PCA-Based Out-of-Sample Extension for Dimensionality Reduction

Dimensionality reduction methods are very common in the field of high dimensional data analysis, where the classical analysis methods are inadequate. Typically, algorithms for dimensionality reduction are computationally expensive. Therefore, their applications to process data warehouses are impractical. It is visible even more when the data is accumulated non-stop. In this paper, an out-of-sample extension scheme for dimensionality reduction is presented. We propose an algorithm which performs an out-of-sample extension to newly-arrived multidimensional data points. Unlike other extension algorithms, such as the Nyström algorithm, the proposed algorithm uses the intrinsic geometry of the data and the properties of dimensionality reduction map. We prove that the error of the proposed algorithm is bounded. Additionally to the out-of-sample extension, the algorithm provides a residual for any new data point that tells us the abnormality degree of this data point.